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(57) Abstract 



A controller, in accordance with a preferred embodiment, provides, in a specified region of first television picture generated by a 
first digitally compressed video signal, an overlay of a second picture embodied in a second video signal. The controller includes an 
I-frame buffer for storing updated macroblock pixel data in I-frame format derived from the second signal. The controller also has a matte 
mask storage register for storing, data that identifies the specified region by means of macroblocks. A matte substitution processor is in 
communication with the matte mask storage register and has as an input the first signal and provides an overlayed output; the controller 
inhibits transmission to the output of any macroblock, of the first signal's data stream, lying in the specified region. It also functions to 
substitute for such macroblock a corresponding macroblock from the I-frame buffer. Further embodiments permit overlay of a graphic 
display for interactive television on a picture derived from a digitally compressed video signal. 
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COMPRESSED DIGITAIi OVERIiAY CONTROLIiER AND METHOD 

5 Technical Field 

The present invention relates to controllers for 
providing in a first television picture generated by signals 
in a compressed digital format, such as under the MPEG2 
standard, an overlay of a second picture. 

10 pacXqyowd Art 

Overlay controllers for analog television signals are 
well known in the art. The MPEG2 compression standard for 
digital video signals also contemplates the possibility of 
overlaying a first compressed digital video signal with a 

15 second compressed digital video signal. A controller for 

achieving the overlay of two such digitally compressed video 
signals is inherently complex. In some instances, it is 
desirable to overlay a compressed digital video signal (such 
as, for example, a movie or other material made available to 

20 a cable television subscriber) with a relatively slowly 
changing analog signal (such as, for example, a display 
including a movable cursor and a number of selection buttons 
to permit interactive cable television service to a 

^ subscriber operating a remote control unit causing movement 

25 of the cursot and dj^eir^liiion bf the buttons) . In su^ 

case, the art currently would require, before overlaying the 
signal, either (i) A-to-D conversion of the analog signal 
and its subsequent compression or (ii) decompression and D- 
to-A conversion of the digital signal. Such techniques are 

30 inefficient and expensive. 

giiwiTn^Y ^ of the Invention ; ■ 

? In accordance with a preferred embodiment of the^ 

, invention there "is provided a* controller, for providing, in 
a specified region of first television picture generated by 

35 a first digitally compressed video signal, an overlay of a: 
second pictT(ire embodied in a sepond video signal. Thtei ; 
controller includes an I-f rame buffer for storing updated 
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macroblock pixel data in' I-fr'aiie format derived frbih the ' 
second' signal. The controller also has i^iriattty mask storage 
register for storing data that identifies We specified ' 
region by means of mkdr'oblocks.- A matte sUbstittition ' 
processor is in coianunication' with the matte "mask sifdrage 
register and has as an input the first signal and provides 
an overlayed output; th'e controiieiir inhibits transmission to 
the output of any mabrobiock/ bf thfeHfiirst signal's d^^ 
stream, lying in the specified r^gibri^^. It also functioni^ to 
substitute for such macroblock a co'rrfespoHding inacrobl6dk 
from the l-frame buffer. Further embbdimehts permit overlay 
of a graphic display for interactive television on a picture 
derived from a. digitally compressed- video ^signal . 

Brief DeseiripCi-ii on-ofrfche DPaw-inqa 
15 ■ The inventibn. will be more i-eadily> understood by- 

reference to the following description taken with the • • 
accompanying drawings in which: ' . 

Fig. i is a di£igra!ai of > thie effect of an overlay in 
accordance with a pr is fferreitJ ? embodiment of the ihV6ntibh; ' 
20 Fig. 2 is a diagraak of a inatW mask used to define the 

boundaries of the overlay in Tig. "1 ' - ■ . - 

FigV 3 is a block diagram of an embodlnient bf the 
invention for achieving the overlay of Fig. i; and ^ ' 
Fig.' 4 provides detail of certain ^portions of Fig. ^ . 

Petailed beserlpiiiir^ tf Off eWg^f fjc Emhbaiin^f-o 
Fig. 1 iPllustrates ?the effect of an bverlay df a' fibrst 
MPEG2 encoded digital video source'; which is referred to 
below as source 1- by a second signal, which is referred to 
below as source 2. In the contekt of interactive - 
30 television; source l iniay bei, for examiile, a dig^^ ' 
compressed iovie or other informatlori service (as that terin 
is defined above) . Source 2 may be, fot example, a graphic 
display, in analog format^ showing a menu of choices 
available to a subscriber atrid a cursor mbvabre by 'the " ^ 
35 subscriber (using a remote cbntroi) , so Arranged ai to 

permit the subscriber to' comiijuhicate iritefactively ' with the 
heaidend and tb . makfe' a selection' from ^thfe meri^ 
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Thus,, source^ l^pould be a. moyie preview,; for example, and. 
source 2 could present a menu of choices to the subscriber, 
permitting selection of .thte movie or, an opportunity, to 
preview another movie., and so ^on. In this example, . sourqe 1 
5 is shown in region of Fig, 1, and source 2 is s;hown in , , 
region B- . _ i. • - ■ 

In accordance with MPEq2^ encoding,, s;uccessive ^ 
contiguous .16 X, 16 square ai^r ays (herein, ca 
"macroblocks") of pixels are , identified pver the entire 

10 picture, area; a sampXe^ macrpb.l^ iS; shown as item 11 in . 
Fig . 1. Each m^croblpqk, in the picture area - is given a . ^ 
unique mimerical; addyressvr :in- the general, manner, illustrated 
in Fig. 1, corresponding dtQ itS; location in the picture. ^ 
The boundary between (regions A an^ B inuFig- 1 is 

15 defined by a matte; miask,, shown; , in ^Fig. 2, tiiat specifies the . 
macroblock* addresses thaj:, are -inqluded :{:or, aJ.ternatively>;L 
excluded) in region A. ; r. u'^ Y > v ^> 

..Fig. 3 , is a block 4i?igrap fpjr j?in- embodiment, of the 
present invention for. a<d:^i<?x4l^g fthe^.^^q^^ described abo:^^e• 

20 Inputs 30, include the, ,^dig.iitaj.ly cpmpresaed ^source 1. signal 
as well as data showing activity; of^ jthe^ subscriber/ s .remote 
unit. (Additipna^l video data may be provided fpr use in 
creating .the video .for regtipn B of Fig.. 1. Such additional 
data may come , as <|igitally/compre^sed^ frpm the inputs 

25 30 and bte .decqmpressed ,^^^ decpmpresspr^33 and/pi: it may 

come from pther source, 34,, wh or TV 

tuner ^ ^ Data frpm ; the subscriber ' s remo:te unit is. ^.processed 
by ,CPU 31r. operatiixg with RAML. 2^^5 , and graphic^ overlay 32, , 
provides an analog input to mixer-encoder 36. This , input 

30 321 shows icursor position^ and the menu choices ayailal^le to 
,the subgprrifcjer ^pr , interact ion with the l&eadend . . The xnixer- 
encqder 3 6 f irS;t: . mixes input - 321 ^with inputs f rom sources 3 3 
and 34, and then^ enqodfLS t^e,. mixed analog signal as .a i series 
of I-fr€Uttei^ in.MEEG2 .f qrniat.. The .I^frame , encoded ^put^ 
35 mixer-rencoder.;>36 is^ provided to MPEG2 , mixer 37 ^ <^^%PP^ 
of. which^is .suLbjectjed to ^.regulation by., flqw^.c^ ^38 to 

prevent excessiv€»ly higb-dat^ rates r. from appear ing at .•t;he - 
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out-put 39. This^ rilgulatiori is riecessary to^^ pr^ the data 

rates from exceeding the capacity of the distribution 
system/ and, particularly> of the' subscriber's e 

Detail of the mixer-entode^^ the MPEG2 mixer 3^, and 

5 the flow controller 38 is presented in Fig. ■ 4. The MPEG2 ' 
encoded digital video data stream identified as source 1 in 
Fig. 4 is here overlay ed with a^desired signal output/ 
identified as source 2 f irom the video Ciffects and mixer 
module 41. in the manner described "herein. In this* example, 

10 soux'ce 2 is in analog YUV format I* Thfe source 2 signal is 
then digitally encoded by encoder 4*2 as a series bhly of I- 
frames in MPEG2 format. (That B-frames or P-frames 

are provided in the output f rdin ^nGbder ' 32 . = it will be ' 
appreciated that the picture Content cf region B is ' 

15 generally expected to be relatively slowly changing; the 
changes most coiomonly expectied^^are primarily those 
associated with cursor laot ion aind' selection of choices by ^ 
the subscriber. Accordingl^yp .the full MPEG encoding of 
'-'source ^2^ *is not-necessary/ v-'^ ^-^r^-i • - ^ (\. > ^ '■. ,?■ 

20 Moreover, * constraints on "the system are required in 

order for it to provide appropriate responsiveness . Each 
frame duration is 1/3 0th of a second. Encoding source 2. as 
a series of I-frames will therefore mean up to l/30th of a 
second until an action on the s.xibscriber's remote unit 

25 (excluding other sources of delay) will have been encoded by 
encoder 42. Moveover, in addition to the time for encoding 
here by encoder 42, there must also be taken into account an 
equal time for decoding for display on the sxibscriber's 
television; thus the total delay is double the time for 

30 encoding. It is possible, for example, to use P frames 

along with I frames, but if P frames are used in addition to 
I frames, the encoding time will be doubled. 

The I-frame output of encoder 42 is stored in I-frame 
buffer 33, the data in which is updated as necessary based 

35 on output from the encoder 42- Any of the macroblocks in 
the I-frame buffer that do not change do not need to be 
updated, and where there is little change in source 2 video 
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there will be. relatively^ the biif fer 

. The I-f rame. data in. buffer 43. is^ selectively, inserted 
in the source 1 MPEG2 encodegl^data stream in the^ following 
5 manner. The source 1 data^ stream is run through, macroblock 
detector 46 to detect maxsrpbloclcs, and its ou:tput is. , 
provided to the matte siabstitutipn prqcesscxr . 44 . Matte mask 
storage register 45, stores ithe addresses of the macroblocks 
constituting the matte masfc.of Fig, 2, jand provides data to 

10 the ma^tte processor 44 V VJhenever the .processor identifies a 
macroblpck (regardless whether. I-, B-r^ or P- frame) from 
detector 46 that lies outside (or, alternatively, , inside) . 
the matte region def ined., 4-n rr,e^ it substitutes 

I-frame macroblock of. thei.,,cQrr^pQnding .address from I-f^rame 

15 buffer .43.. : - . v.i.sT r:.. - - . 

It is possfible, if there, is .a .gxeat deal qf change, in 
the^ picture content ot . $puz:ce , Z , then thei ..output -39 ^ of . ^ 
system. would be at a , data nat^ beyond system capacity. In 
such a case, the processor 44 will j continue ;to block source 

20 1 macroblocks lying outside the matte region, but will 

refrain from substi^tuting data from, the I-frame buffer 43 to 
rthe extent necessary to slow the. data rate at the output to 
a rate within systeu/ limits. » : ; > . : 




9533342A1_I_> 



W6 95/33342 



PCTAJS95/06924 



- € - 

Mlat Claimed 'is: , ^ ■^'•-•^ / .w^.r i.^ . ' : 

r. A cohtrdller, for providing/ in~'i^ spebified region 
of jet television picture generated a firsit digitally 
compressed video signal', an overlay of a secbhd picture 
5 embodied in a second video signal > comprisihg: 

- an I-frame buffer for storing updated macroi3lock pixel 
data in I -frame foridat derived frdtf second sighcil; ^ 
miatte mask storage regiister for storing data that ^ 
identifies the specified region by mfeans of macrbblodks; 
10 matte substitution processor in ^ communication with the 

matte- maisk storage register' arid havinig as an input the first 
signal and providing an overlayed output/ for inhibiting 
transmission td the output of^ ahy^macrbblbck; b the first 
signal's data stream, lying iK^tKe^ spebified'^^i^^^ and for 

15 sxabstituting for such meicrbblbck- a* corrfesi^onding macroblock 
from the I-frame buffer", * ^ 

2. A controller according to claim 1, wherein the 
second! video: signal^ isiiin ari^ ahalog fom further 
comprising :. \ --v.-x: vi^n^-^ - : s v o '.r^ - -f- • 

20 an 1-frame : encoder for enbodiiig the Second 'Signal a^ a 

series of I^frames, the I -tirame encoder having ^n output iri 
communication with the I-frame buffer. 

3, A controller according to claim 2/ further 
comprising: 

25 a macroblock detector in communication with the first 

signal and the matte substitution processor/ for * detecting 
the preisence of a macroblock. ' - ^ - 

A. A controller according to claim 1, Wherein the 
matte substitution processor includes flow con^ means for 

30 refraining from sxibstittiting data' from the I-frame buffer, 
while continuing to inhibit trankmiissxori of any macrob 
of the first sigriial's data stream lying in the specified ' 
region, to the extent necessary to slow thb data i-ate at the 
overlayed output to a rate within system limits. ' 

35 5 . A controller according to claim" 3 , wherein the 

matte substitution prbcessor includes flow cbntrol means for 
refraining from siibsti'ttiting data firom^ i-frkme buffer. 
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while continuing to inhibit transmission ,pf any macrobloek 
of the first ,^signa,L's data .stream lying in the specified 
region, to the extent necess.ary to, slow, the data rate at the 
overlayed output to a^ rate within sys^tem limits* ; . 
5 6, An interactive television system .providing,; in a 

specified, region of a television picture generated by, a 
first digitally compressed video signal, an .pyerlay of a , 
graphic display permitting user, inpiut ,via. a remote control , 
such graphic display embpdied iin a second signal in an , ^ 
10 analog foirmat, the s;ystem comprising: 

an I- frame. encoder for encoding the second signal as a 
series of I-frames;^ , i . ? . _ . , : i 

an I-f rame .buf f er f pr ^st^ macroblpck pixel 

data in I-frame format ^^from. tl^e I?-.frame encoder; 



15 mat:te mask s^prage r<sgister, for storing » dat^ ^ 4 

identifies the specified region by means of macroblocks;- 

matte sxibstitution processor ;, , in communication with the .at 
matte mask storage registei;>and/ hav^ as an input the first tr 
signal and providing an overlayed output, for inhibiting 

20 transmission to the pvitput spf i,any macroblock^j ^of the first v 

signal's data stream, lying in the specif led region, andr for ^n^^ 
substituting for such macrpblpck'a corresponding macroblock . 



from the I-frame buffer. 

?• A controller according to claim 6, further 
25 comprisii^g:.. . : - . .ij.a^. ■/ J--^ • ^ ■ 

a macroblock detector , in communication with the first 
signal and the matte substitutipn prppessor, for detiecting 
the presence of a macroblock, 

8. A controller according, to claim 6, wherein the 
30 matte substitution processor incl^ control means for 

refraining tirom substit^i^tingr data frpm the I-frame buffer, 
whil^ continuing to i^nhibit transmlssipn of any , macroblock 
of the first signal's, data stream lying in 

region, to the^ exte^rit necessary.^ tp; slow t rat^B at. the 

35 overlayed out^put to j a. rate within system ,limit?5. 

, 9 . A # cqritrpller^, a,ccprding to , clajLm , 7 # • wherein the : 
matte s^bst i^ution prpcessor^ i^ f Ipw, qpntrol means for 
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refraining from substituting data ? from the I-frame buffer, 
while continuing to; inhibit transmission of :>&ny macroblock ■ 
of the first signal ' s data stream -lying > in the specif ied 
region, to the extent necessary to slow the data rate^ at the 
overlayeti output to a rate within^ ^system limits. 

10. A method of overlaying in a specified region of a 
television picture a' first digitally compressed video: signal 
with a second analog video signal, comprising: 

encoding the s<iicond signal as^ a ^ series of I-frames; 

storing the encoded i-frame macroblock pixel data in an 
I-frame buffer; r ' 

storing data that identifies j the specif ied region by 
means' of macroblocks in a matte ^mask ^ storage register ; ^ 

processing the data stream of the first signal and 
providing an overlayed output, by inhibiting transmission to 
the output of any macroblock, of the first signal's data 
stream, lying in the specified region and substituting for 
such macroblock a corresponding macroblock from the I-frame 
buffer. 

11. A method according to claim 10, further 
comprising: 

refraining from substituting data from the I-frame 
buffer, while continuing to inhibit transmission of any 
macroblock of the first signal's data stream lying in the 
specified region, to the extent necessary to slow the data 
rate at the overlayed output to a rate within system limits. 

12. A method of providing an interactive television 
system having, in a specified region of a television picture 
generated by a first digitally compressed video signal, an 
overlay of a graphic display permitting user input via a 
remote control, such graphic display embodied in a second 
signal in an analog format, the method comprising: 

encoding the second signal as a series of I-frames; 
storing the encoded I-frame macroblock pixel data in an 
I-frame buffer; 

storing data that identifies the specified region by 
means of macroblocks in a matte mask storage register; 
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processing -the data Stream Of the first signal, and . - 
providing anyoverlayed ^output , by inhibiting transmission to 
the output of any macroblocKv of the first signal f s data ; 
,i stream, lying in; the specif ied .region and; substituting . for 
5 such macroblock a. corresponding, macroblock from the I-frame 
buffer. - , . : ■. . V • A. ■ : ■ - ; ..o . . ' 

, v.i 13. A method acco^-ding to, claim. 12,,. f;urther , ^ 
comprising: , ^ > , , r^.- , ^ . ; . 

refraining from, substituting data from the . I-frame 
10 buffer, . while/ continuing? to inhibit transmission. of any 

macroblock of the first signal's data stream lying in .:the - 
specified region,^ to, thev extent necessary to slow the data 
rate at the overlayed , otitpwt to a rate within system, limits. 



15 
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